A quasi-experimental evaluation of an on-line formative assessment and tutoring system.

ASSISTment is a web-based math tutor designed to address the need for timely student assessment while simultaneously providing instruction, thereby avoiding lost instruction time that typically occurs during assessment. This paper presents a quasi-experiment that evaluates whether ASSISTment use has an effect on improving middle school students\u27 year-end test scores. The data was collected from 1240 seventh graders in three treatment schools and one comparison school. Posttest (7th grade year-end test) results indicate, after adjusting for the pretest (6th grade year-end test), that students in the treatment schools significantly outperformed students in the comparison school and the difference was especially present for special education students. A usage analysis reveals that greater student use of ASSISTments is associated with greater learning consistent with the hypothesis that it is useful as a tutoring system. We also 2 found evidence consistent with the hypothesis that teachers adapt their whole class instruction based on overall student performance in ASSISTments. Namely, increased teacher use (i.e., having more students use the system more often) is associated with greater learning among students with little or no use suggesting that those students may have benefited from teachers adapting their whole-class instruction based on what they learned from ASSISTment use reports. These results indicate potential for using technology to provide students instruction during assessment and to give teachers fast and continuous feedback on student progress

Knowledge Engineering for Intelligent Tutoring Systems: Using machine learning assistance to help humans tag questions to skills based upon the words in the questions.

Building a mapping between items and their related knowledge components, while difficult and time consuming, is central to the task of developing affective intelligent tutoring systems. Improving performance on this task by creating a semi-automatic skill encoding system would facilitate the development of such systems. The goal of this project is to explore techniques involved in text classification to the end of improving the time required to correctly tag items with their associated skills

Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems

A major issue in Intelligent Tutoring Systems is off-task student behavior, especially performance-based gaming, where students systematically exploit tutor behavior in order to advance through a curriculum quickly and easily, with as little active thought directed at the educational content as possible. The goal of this research was to develop a passive visual indicator to deter and prevent off-task gaming behavior without active intervention, via graphical feedback to the student and teachers. Traditional active intervention approaches were also constructed for comparison purposes. Our passive graphical intervention has been well received by teachers, and results suggest that this technique is effective at reducing off-task gaming behavior

A Methodology for Evaluating Predictions of Transfer and an Empirical Application to Data from a Web-Based Intelligent Tutoring System: How to Improve Knowledge Tracing in Dialog Based Tutors

Cognitive Science is interested in being able to develop methodologies for analyzing human learning and performance data. Intelligent tutoring systems need good cognitive models that can predict student performance. Cognitive models of human processing are also useful in tutoring because well-designed curriculums need to understand the common components of knowledge that students need to be able to employ (cite Koedinger paper and algebra stuff). A common concern is being able to predict when transfer should happen. We describe a methodology (first used by Koedinger, 2001) that uses empirical data and cognitively principled task analysis to evaluate the fit of cognitive models. This methodology seems particularly useful when you are trying to find evidence for “hidden” knowledge components that are hard to assess because they are confounded with accessing other knowledge components. We present this methodology as well as an illustration showing how we are trying to use this method to answer an important cognitive science issue

Ensembling predictions of student post-test scores for an intelligent tutoring system.

________________________________________________________________________ Over the last few decades, there have been a rich variety of approaches towards modeling student knowledge and skill within interactive learning environments. There have recently been several empirical comparisons as to which types of student models are better at predicting future performance, both within and outside of the interactive learning environment. A recent paper (Baker et al., in press) considers whether ensembling can produce better prediction than individual models, when ensembling is performed at the level of predictions of performance within the tutor. However, better performance was not achieved for predicting the post-test. In this paper, we investigate ensembling at the post-test level, to see if this approach can produce better prediction of post-test scores within the context of a Cognitive Tutor for Genetics. We find no improvement for ensembling over the best individual models and we consider possible explanations for this finding, including the limited size of the data set

Pedagogical Agents for Fostering Question-Asking Skills in Children

Question asking is an important tool for constructing academic knowledge, and a self-reinforcing driver of curiosity. However, research has found that question asking is infrequent in the classroom and children's questions are often superficial, lacking deep reasoning. In this work, we developed a pedagogical agent that encourages children to ask divergent-thinking questions, a more complex form of questions that is associated with curiosity. We conducted a study with 95 fifth grade students, who interacted with an agent that encourages either convergent-thinking or divergent-thinking questions. Results showed that both interventions increased the number of divergent-thinking questions and the fluency of question asking, while they did not significantly alter children's perception of curiosity despite their high intrinsic motivation scores. In addition, children's curiosity trait has a mediating effect on question asking under the divergent-thinking agent, suggesting that question-asking interventions must be personalized to each student based on their tendency to be curious.Comment: Accepted at CHI 202

Towards an Intelligent Tutor for Mathematical Proofs

Computer-supported learning is an increasingly important form of study since it allows for independent learning and individualized instruction. In this paper, we discuss a novel approach to developing an intelligent tutoring system for teaching textbook-style mathematical proofs. We characterize the particularities of the domain and discuss common ITS design models. Our approach is motivated by phenomena found in a corpus of tutorial dialogs that were collected in a Wizard-of-Oz experiment. We show how an intelligent tutor for textbook-style mathematical proofs can be built on top of an adapted assertion-level proof assistant by reusing representations and proof search strategies originally developed for automated and interactive theorem proving. The resulting prototype was successfully evaluated on a corpus of tutorial dialogs and yields good results.Comment: In Proceedings THedu'11, arXiv:1202.453